Transformations enabling to construct limited-memory Broyden class methods
نویسندگان
چکیده
The Broyden class of quasi-Newton updates for inverse Hessian approximation are transformed to the formal BFGS update, which makes possible to generalize the well-known Nocedal method based on the Strang recurrences to the scaled limited-memory Broyden family, using the same number of stored vectors as for the limited-memory BFGS method. Two variants are given, the simpler of them does not require any additional matrix by vector multiplications. Numerical results indicate that this approach can save computational time.
منابع مشابه
Generalizations of the limited-memory BFGS method based on the quasi-product form of update
Two families of limited-memory variable metric or quasi-Newton methods for unconstrained minimization based on quasi-product form of update are derived. As for the first family, four variants how to utilize the Strang recurrences for the Broyden class of variable metric updates are investigated; three of them use the same number of stored vectors as the limitedmemory BFGS method. Moreover, one ...
متن کاملOn Solving Large-scale Limited-memory Quasi-newton Equations
We consider the problem of solving linear systems of equations with limited-memory members of the restricted Broyden class and symmetric rank-one matrices. In this paper, we present various methods for solving these linear systems, and propose a new approach based on a practical implementation of the compact representation for the inverse of these limited-memory matrices. Using the proposed app...
متن کاملBFGS with Update Skipping and Varying Memory
We give conditions under which limited-memory quasi-Newton methods with exact line searches will terminate in n steps when minimizing n-dimensional quadratic functions. We show that although all Broyden family methods terminate in n steps in their full-memory versions, only BFGS does so with limited-memory. Additionally, we show that full-memory Broyden family methods with exact line searches t...
متن کاملLimited-memory projective variable metric methods for unconstrained minimization
A new family of limited-memory variable metric or quasi-Newton methods for unconstrained minimization is given. The methods are based on a positive definite inverse Hessian approximation in the form of the sum of identity matrix and two low rank matrices, obtained by the standard scaled Broyden class update. To reduce the rank of matrices, various projections are used. Numerical experience is e...
متن کاملVariable Metric Stochastic Approximation Theory
We provide a variable metric stochastic approximation theory. In doing so, we provide a convergence theory for a large class of online variable metric methods including the recently introduced online versions of the BFGS algorithm and its limited-memory LBFGS variant. We also discuss the implications of our results in the areas of eliciting properties of distributions using prediction markets a...
متن کامل